Systems, methods, and mediums for producing panoramic images

ABSTRACT

The present disclosure describes systems and methods for producing panoramic images. The systems and methods may include storing a first image in a memory device, overlapping a plurality of sections of the stored first image over corresponding portions of a second image such that the second image is visible between adjacent pairs of the plurality of sections in response to the plurality of sections and the second image being displayed on a display device, and aligning at least one of the plurality of sections of the first image with at least a portion of the second image prior to storing the second image.

TECHNICAL FIELD

The present disclosure is related to systems, methods, and mediums forproducing images, such as panoramic images.

BACKGROUND

Known digital cameras are configured to produce panoramic images bydividing a scene in a plurality of images sequentially captured in onedirection. Each of the plurality of images includes an overlappingsection at an edge of the image that allows adjacent pairs of the imagesto be aligned to form the panoramic image. One disadvantage of knowndigital cameras is that the overlapping section may obscure a portion ofthe image that otherwise may be useful with respect to capturing alarger portion of the scene. Another disadvantage of known digitalcameras is that a luminance, a chrominance, or a hue may change betweensequential images because such sequential images are captured atdifferent times and may be captured at different settings. Such changesadversely may affect the resulting panoramic image when adjacent pairsof the images are aligned to form the panoramic image.

BRIEF DRAWINGS DESCRIPTION

FIG. 1 depicts a block diagram of an exemplary system configured toproduce panoramic images.

FIG. 2 depicts an example of a panoramic image including a plurality ofsequential images.

FIG. 3 depicts an example of an overlapping section of the image shownon FIG. 2, used to align sequential images.

FIG. 4 depicts an example of a panoramic image including a plurality ofsequential images.

FIG. 5 depicts an example of a plurality of overlapping sections in theimage shown in FIG. 4, used to align sequential images.

FIG. 6 depicts an example of an automatic exposure adjustment ofsequential images.

FIG. 7 depicts an example of a panoramic image, as displayed on aviewfinder.

FIG. 8 depicts an example of an exposure adjustment of the panoramicimage shown in FIG. 7.

DETAILED DESCRIPTION

FIG. 1 is a block diagram of an exemplary system 100 that includes aninput 110, a memory 120, a controller 130, an image processor 135, animage sensor 140, a network interface 150, and an output 160.

Input 110 may include any mechanism that is configured to provide inputto or to control system 100 or controller 130. Input 110 may include anymechanism, such as buttons, toggle switches, wheels, touch screens,microphones, actuators, or the like, or any combinations thereof, thatis configured to be enabled or actuated, either manually by a user orautomatically by system 100, in response to signals, indications,stimuli, or the like, or any combination thereof.

Memory 120 may store instructions that, in response to execution bycontroller 130, cause controller 130 to perform operations for producingpanoramic images. Memory 120 also may store still images or videos oflive subjects. Memory 120 may include volatile and non-volatile memorydevices, such as Random Access Memory (RAM), Dynamic Random AccessMemory (DRAM), Static Random Access Memory (SRAM), flash memory, ReadOnly Memory (ROM), or the like, or any combination thereof.

Controller 130 may include one or more processing devices configured tooperate or to control system 100. Controller 130 may be configured toread the instructions stored in memory 120, and to configure system 100in a variety of ways.

Image sensor 140 may comprise a grid of capture devices, e.g.,capacitors or transistors (not shown). Image sensor 140 may beconfigured to sense an image by converting light focused on the imageinto electrons that charge the capture devices. The more intense thelight is at each capture device, the greater the charge accumulated inthe capture device. Image processor 135 may read the charge at eachcapture device to generate a digital value representing brightness of acorresponding pixel in the image. Image processor 135 may include adevice integral to controller 130 or a standalone device separate fromcontroller 130. Image sensor 140 also may include a Charge CoupledDevice (CCD) or Complementary Metal Oxide Semiconductor (CMOS) device.

To obtain a color image, system 100 also may include a beam splitter(not shown) configured to split the light into red, blue, and greencomponents and to send each component to a corresponding grid of capturedevices. One of ordinary skill in the art readily will understand thatother methods of recording color in an image, such as using filtersalone or in combination with algorithms, other circuits, or anycombination thereof, may be employed.

Network interface 150 may connect, couple, or otherwise interface withsystem 100 via a wired or a wireless network. Network interface 150 mayenable uploading of an image from system 100 to the wired or wirelessnetwork or enable downloading of images or other files or data,executable or otherwise, from the wired or wireless network. Networkinterface 150 further may enable updating firmware, drivers, and/or thelike associated with system 100.

Output 160 may include any mechanism configured to allow system 100 toprovide output to the wired or wireless networks, users, or the like, orany combination thereof. Output 160 may include a display configured toindicate a status of various settings of system 100, to display one ormore of the images making up the panoramic image, or the like, or anycombination thereof. Output 160 also may include speakers to generatesound, such as sound associated with various settings or status ofsystem 100.

One of ordinary skill in the art readily will understand that a varietyof devices, such as cameras, personal digital assistants, cellularphones, computers, or the like, and/or any other device, handheld orotherwise, digital or otherwise, that are capable of capturing an image,may be used to implement examples within contemplation of system 100.

FIG. 2 depicts an example of a panoramic image 200. Panoramic image 200may include a first image 210 overlapping a second image 230. Anoverlapping column 220 may exist between, and be common to, first image210 and second image 230. Overlapping column 220 may extend the rightlength of first image 210 and overlap the left length of second image230, such that overlapping column 220 facilitates a user's alignment offirst image 210 with second image 230 to generate panoramic image 200. Auser may capture first image 210 and second image 230 sequentially asthe user moves the device implementing system 100 from a first directionto a second direction, e.g., from left to right, right to left, up todown, down to up, or the like, or any combination thereof.

FIG. 3 depicts an example of an overlapping column 220 of the imageshown in FIG. 2, used to align sequential images. Referring to FIGS.1-3, system 100 may produce panoramic image 200 in response to apanoramic image mode set by a user actuating any of inputs 110. Whensystem 100 is in the panoramic image mode, system 100 automatically maydisplay a portion of first image 210 that will overlap with second image230 to enable visual alignment of the first image 210 and the secondimage 230. System 100 may display overlapping column 220 on output 160,e.g., the left length of the viewfinder of a device, after capturingfirst image 210. As such, system 100 may allow a user to alignoverlapping column 220 of the first image 210 with a portion 240 ofsecond image 230. The user then may move the camera in variousdirections better to ensure that overlapping column 220 properly isaligned with portion 240.

In an example, system 100 may be configured to produce a panoramic image400, as shown in FIG. 4. Referring to FIGS. 1, 4, and 5, panoramic image400 may include a first image 410 and a second image 430. A user of adevice implementing system 100 may capture first image 410 in memory 120and may display second image 430 on the output 160, e.g., a device'sviewfinder (not shown), prior to capturing second image 430 in memory120.

System 100 may display a plurality of sections 420A, 420B, and 420C offirst image 410 overlapping second image 430 on output 160. Sections420A, 420B, and 420C may overlap second image 430 such that at least aportion of second image 430 is visible between adjacent pairs ofsections 420A, 420B, and 420C. FIGS. 4 and 5 show three sections 420A,420B, and 420C, however, one of ordinary skill in the art readily willunderstand that examples including two sections and examples includingmore than three sections also are within the scope of the presentdisclosure. Second image 430 may be visible between adjacent pairs ofplurality of sections 420A, 420B, and 420C, e.g., section 420A andsection 420B, which may facilitate or enable a user's visual alignmentof second image 430 with first image 410 in one or more axes, e.g., anx-axis, a y-axis, or a z-axis. As with image 200, a user may capturefirst image 410 and second image 430 sequentially as the user moves thedevice implementing system 100 from one direction to another direction,e.g., from a left to right direction, from a right to left direction,from an up to down direction, from a down to up direction, or anycombination thereof.

System 100 may display first image 410, second image 430, or panoramicimage 400, or any combination thereof, on output 160, e.g., the device'sviewfinder.

In an example, system 100 may enable a user to manually adjust theexposure between first image 410 and second image 430 using anadjustment bar 450 alone or in combination with input 110. For example,the user may actuate an exposure adjustment by depressing a button onthe device that increases or decreases the exposure of first image 410,second image 430, or sections 420A, 420B, and 420C, or any combinationthereof. System 100 graphically may show the exposure adjustment bydisplaying adjustment bar 450 on output 160, e.g., the device'sviewfinder. Alternatively, adjustment bar 450 may allow adjustmentdirectly, e.g., with a graphical display of adjustment bar 450 on thedevice's screen that responds to a user's touch actuation. When system100 is in a fixed shutter speed mode, system 100 may allow a user tochange the aperture. When system 100 is in a fixed aperture mode, system100 may allow the user to change the shutter speed.

System 100 may enable automatic exposure adjustment between first image410 and second image 430. Referring to FIGS. 1 and 4-6, system 100 maydivide a portion of first image 610 into a plurality of firstsubsections 611-619 and an overlapping portion of second image 620 intoa plurality of second subsections 621-629. System 100 may or may notdisplay the first subsections 611-619 or the second subsections 621-629on output 160. In an example, corresponding pairs of first subsections611-619 and second subsections 621-629 of first image 410 and secondimage 430, respectively, may be substantially identical in size. System100 may measure a size of first subsections 611-619 and a size of secondsubsections 621-629 in pixels. System 100, e.g., controller 130 or imageprocessor 135 of system 100, may calculate a brightness of each of firstsubsections 611-619 and each of second subsections 621-629. System 100automatically may adjust an exposure of each of first subsections611-619, each of second subsections 621-629, or any combination thereof,in response to comparing each of first subsections 611-619 to acorresponding one of second subsections 621-629. In one embodiment,system 100 automatically may adjust the exposure of each of firstsubsections 611-619, each of second subsections 621-629, or anycombination thereof, by minimizing a difference in brightness betweeneach of the plurality of first subsections 611-619 and the correspondingone of second subsections 621-629. For example, system 100 may calculatethe brightness of pixels in first subsection 611 and the brightness ofpixels in second subsection 621. In response to calculating a differencein brightness levels between first subsection 611 and correspondingsecond subsection 621, system 100 may adjust up or down the brightnessfor pixels in first subsection 611, the brightness for pixels in secondsubsection 621, or any combination thereof. System 100 also maycalculate a brightness of individual pixels or an average brightness forpixels within first subsections 611-619 and/or with second subsections621-629. Moreover, system 100 automatically may adjust a brightness ofindividual pixels or automatically may adjust a brightness of groups ofpixels.

Based on the foregoing, in one example, the brightness for each of firstsubsections 611-619 may correspond to an average brightness of pixels infirst subsection 611-619, respectively, and may be represented by B_(I).In this example, the brightness for each of second subsections 621-629may correspond to an average brightness of pixels in second subsection621-629, respectively, and may be represented by B_(J). Moreover, adifference in brightness levels between first subsections 611-619 andsecond subsections 621-629 may be represented by the formulaΣ|(B_(I)−B_(J))| or by the formula Σ(B_(I)−B_(J))².

System 100 automatically may adjust an exposure of second image 430 tominimize the difference between corresponding pairs of first subsections611-619 and second subsections 621-629, e.g., the difference betweenfirst subsection 611 and second subsection 621.

System 100 may or may not similarly size first section 610 and/or secondsection 620 with sections 420A, 420B, and 420C, for example, becausefirst section 610 or second section 620 and/or sections 420A, 420B, and420C may serve different purposes. System 100 may use first section 610and second section 620 to automatically adjust exposure between twosequential images, e.g., first image 410 and second image 430. System100 may use sections 420A, 420B, and 420C to enable aligning first image410 with second image 430. System 100 may display or otherwise may makevisible sections 420A, 420B, and 420C on output 160 to facilitatealignment. System 100 may not display first section 610 and/or secondsection 620 because system 100 may use first section 610 and/or secondsection 620 internally.

In an example, system 100 may assemble sequential images, e.g., firstimage 410 and image 430 and also may allow a user to adjust theluminance, the chrominance, the hue, or any combination thereof(collectively LCH), of each image individually or as displayed assembledas panoramic image 400. System 100 also may enable a user to switchdisplay views from panoramic image 400 to individual images, e.g., firstimage 410 and/or second image 430 in response to actuation of input 110.

FIG. 7 depicts an example of a panoramic image 720, as displayed on adisplay. Referring to FIGS. 1 and 4-7, system 100 may display the resultof assembling sequential images that make up panoramic view 700. System100 may display the panoramic image 720 with an indication of boundariesbetween individual sequential images using a boundary bar 710. In anexample, system 100 also may indicate the boundaries between individualsequential images using lines or by identifying the individualsequential images with numbers, characters, or the like, or anycombination thereof. System 100 further may enable a user to viewpanoramic image 720 on output 160 and may enable adjusting the LCH ofeach image to match adjacent images.

FIG. 8 depicts an example of an exposure adjustment of a panoramic image800, which may be similar to the image shown in FIG. 7. Referring toFIGS. 1 and 4-8, system 100 may display panoramic image 800 on output160, e.g., a device's display, together with a first boundary bar 810and a second boundary bar 820 indicating the boundaries betweenindividual images 811 and 821 making up the panoramic image 800. System100 additionally may enable a user to move an image displayed on output160 by actuating a first arrow 850 or a second arrow 860 using input110, e.g., a device button. For example, a main portion of the panoramicimage 800 is shown as image 821, as indicated by boundary bar 820.System 100 may enable a user to adjust settings of the individual image1 and image 2. For example, system 100 may enable a user to adjust abrightness of image 2 by actuating adjustment bar 870 using input 110.System 100 also may enable a user to select which component of image 821the user wants to adjust by changing input 110. For example, in responseto the user selecting to adjust luminance, adjustment bar 870 mayreflect luminance of the image 821. System 100 also may enablereplacement of an individual image with other, non-sequential, images.

The invention claimed is:
 1. A method, comprising: capturing a firstimage in a memory device; before capturing a second image in the memorydevice: substantially simultaneously displaying a plurality of sectionsof the stored first image overlapping the second image such that thesecond image is visible between pairs of the plurality of sections;enabling aligning at least one of the plurality of sections of the firstimage with at least a corresponding portion of the second image; andautomatically adjusting an exposure setting of each of the plurality ofsections of the first image, each of a plurality of sections of thesecond image, or any combination thereof in response to comparing eachof the plurality of sections of the first image to a corresponding oneof the plurality of sections of the second image to minimize adifference in brightness between each of the plurality of sections ofthe first image and the corresponding one of the plurality of sectionsof the second image; and capturing the second image in the memory deviceusing the automatically adjusted exposure setting.
 2. The method ofclaim 1, further comprising enabling manual adjustment of the exposuresetting via an interactive adjustment bar displayed on the displaydevice overlapping the first image or the second image.
 3. The method ofclaim 1, further comprising automatically changing an aperture settingin response to a shutter speed being fixed or automatically changing theshutter speed in response to the aperture setting being fixed.
 4. Themethod of claim 1, wherein automatically adjusting the exposure settingfurther comprises: identifying a plurality of first subsections in thefirst image and a plurality of second subsections in the second image;comparing each of the plurality of first subsections to a correspondingone of the plurality of second subsections; and automatically adjustingthe exposure setting between the first image and the second image basedon a result of the comparison prior to storing the second image.
 5. Themethod of claim 4, wherein comparing each of the plurality of firstsubsections to the corresponding one of the second subsections includesminimizing a difference in brightness between each of the plurality offirst subsections and the corresponding one of the second subsections.6. The method of claim 1, further comprising: generating a compositeimage by assembling the first image and the second image in response tothe aligning after capturing the second image in the memory device; anddisplaying the composite image on the display device.
 7. The method ofclaim 6, further comprising: enabling adjustment of at least one ofluminance, chrominance, or hue associated with the first image or thesecond image in response to displaying the composite image on thedisplay device.
 8. The method of claim 6, further comprising: enablingreplacement of the first image or the second image with a third image inresponse to the displaying.
 9. The method of claim 6, wherein displayingincludes displaying the composite image in response to a shutter releaseassociated with capturing the second image.
 10. An apparatus,comprising: a memory device configured to store a first image capturedby an image sensor in response to a first shutter release; a displaydevice configured to substantially simultaneously display a plurality ofsections of the first image over at least portions of a second imagesuch that the second image is visible only between adjacent pairs of theplurality of sections of the first image; and an image processing deviceconfigured to: enable alignment of at least one of the plurality ofsections of the first image with at least a corresponding portion of thesecond image by allowing movement of the second image while the firstimage remains fixed and overlapping the second image; automaticallyadjusting an exposure setting of each of a plurality of sections of thesecond image in response to comparing the at least one of the pluralityof sections of the first image with at least one of the plurality ofsections of the second image to minimize a difference in brightnessbetween the at least one of the plurality of sections of the first imageand the at least one of the plurality of sections of the second image;and storing the second image in the memory device using theautomatically adjusted exposure setting in response to a second shutterrelease.
 11. The apparatus of claim 10, wherein the image sensorcomprises a charge-coupled device.
 12. The apparatus of claim 10,further comprising a network interface configured to interface with atleast one of a wireless or a wired network.
 13. The apparatus of claim10, wherein the image processing device is further configured to enablealignment of at least one of the plurality of sections of the firstimage with the at least the corresponding portion of the second image inresponse to movement of the apparatus.
 14. The apparatus of claim 10,wherein the image processing device is further configured to enablemanual adjustment of the exposure setting using an interactiveadjustment bar displayed on the display device.
 15. The apparatus ofclaim 10, wherein the image processing device is further configured toautomatically change an aperture setting in response to a shutter speedbeing fixed or to automatically change the shutter speed in response tothe aperture setting being fixed.
 16. The apparatus of claim 10, whereinthe image processing device is further configured to: identify aplurality of first subsections in the first image and a plurality ofsecond subsections in the second image; compare each of the plurality offirst subsections to a corresponding one of the second subsections; andautomatically adjust the exposure setting between the first image andthe second image based on a result of the comparison prior to storingthe second image in the memory device.
 17. The apparatus of claim 16,wherein the image processing device is further configured to minimize adifference in brightness between each of the plurality of firstsubsections and the corresponding one of the second subsections.
 18. Theapparatus of claim 10, wherein the image processing device is furtherconfigured to: generate a composite image by assembling the first imageand the second image in response to the alignment; and display thecomposite image on the display device.
 19. The apparatus of claim 18,wherein the image processing device is further configured to: enableadjustment of at least one of luminance, chrominance, or hue associatedwith the first image or the second image in response to displaying thecomposite image on the display device.
 20. The apparatus of claim 18,wherein the image processing device is further configured to: enablereplacement of the first image or the second image with a third image inresponse to displaying the composite image on the display device.
 21. Anapparatus, comprising: a memory device configured to: storeinstructions; store a first image in response to a first shutterrelease; and a processing device configured to execute the instructionsstored in the memory device to: substantially simultaneously display aplurality of sections of the first image over at least portions of asecond image such that the second image is visible only between adjacentpairs of the plurality of sections of the first image; enable alignmentof at least one of the plurality of sections of the first image with atleast a corresponding portion of the second image by allowing movementof the second image while the first image remains fixed and overlappingthe second image; automatically adjust an exposure setting of each of aplurality of sections of the second image in response to comparing theat least one of the plurality of sections of the first image with atleast one of the plurality of sections of the second image to minimize adifference in brightness between the at least one of the plurality ofsections of the first image and the at least one of the plurality ofsections of the second image; and wherein the memory device isconfigured to store the second image using the automatically adjustedexposure setting in response to a second shutter release.
 22. Theapparatus of claim 21, wherein the processing device is configured toexecute the instructions stored in the memory device further to: enablemanual adjustment of the exposure setting between the first image andthe second image in response to comparing at least one of the pluralityof sections with the corresponding portion of the second image.
 23. Theapparatus of claim 22, wherein the processing device is configured toexecute the instructions stored in the memory device further to: enablemanual adjustment of the exposure setting via an interactive adjustmentbar displayed on the display device.
 24. The apparatus of claim 22,wherein the processing device is configured to execute the instructionsstored in the memory device further to: automatically change an aperturesetting in response to a shutter speed being fixed or automaticallychange the shutter speed in response to the aperture setting beingfixed.
 25. The apparatus of claim 21, wherein the processing device isconfigured to execute the instructions stored in the memory devicefurther to: identify a plurality of first subsections in the first imageand a plurality of second subsections in the second image; compare eachof the plurality of first subsections to a corresponding one of theplurality of second subsections; and automatically adjust the exposuresetting between the first image and the second image based on thecomparison.
 26. The apparatus of claim 25, wherein the processing deviceis configured to execute the instructions stored in the memory devicefurther to: minimize a difference in brightness between each of theplurality of first subsections and the corresponding one of the secondsubsections.
 27. The apparatus of claim 21, wherein the processingdevice is configured to execute the instructions stored in the memorydevice further to: generate a composite image by assembling the firstimage and the second image in response to the aligning; and enabledisplay of the composite image on the display device.
 28. The apparatusof claim 27, wherein the processing device is configured to execute theinstructions stored in the memory device further to: adjust at least oneof luminance, chrominance, or hue associated with the first image or atleast portions of the second image in response to displaying thecomposite image on the display device.
 29. The apparatus of claim 27,wherein the processing device is configured to execute the instructionsstored in the memory device further to: enable replacement of the firstimage or the second image with a third image in response to displayingthe composite image on the display device.
 30. The apparatus of claim27, wherein the processing device is configured to execute theinstructions stored in the memory device further to: enable display ofthe composite image in response to the second shutter release associatedwith the storing of the second image.
 31. A computer readable storagedevice comprising executable instructions stored thereon that configurea processing device to perform operations comprising: storing a firstimage in a memory device in response to a first shutter release;substantially simultaneously displaying a plurality of sections of thestored first image overlapping a second image such that the second imageis visible between pairs of the plurality of sections; enabling aligningat least one of the plurality of sections of the first image with acorresponding portion of the second image by allowing movement of thesecond image while the first image remains fixed and overlapping thesecond image; automatically adjusting an exposure setting of each of aplurality of sections of the second image in response to comparing theat least one of the plurality of sections of the first image with atleast one of the plurality of sections of the second image to minimize adifference in brightness between the at least one of the plurality ofsections of the first image and the at least one of the plurality ofsections of the second image; and storing the second image in the memorydevice using the automatically adjusted exposure setting in response toa second shutter release.
 32. The computer readable storage device ofclaim 31, wherein the processing device is configured to performoperations further comprising: enabling manual adjustment of theexposure setting between the first image and the second image inresponse to comparing at least one of the plurality of sections with thecorresponding portion of the second image.
 33. The computer readablestorage device of claim 32, wherein the processing device is configuredto perform operations further comprising: enabling manual adjustment ofthe exposure setting via an interactive adjustment bar displayed on thedisplay device.
 34. The computer readable storage device of claim 32,wherein the processing device is configured to perform operationsfurther comprising: automatically changing an aperture setting inresponse to a shutter speed being fixed or automatically changing theshutter speed in response to the aperture setting being fixed.
 35. Thecomputer readable storage device of claim 31, wherein the processingdevice is configured to perform operations further comprising:identifying a plurality of first subsections in the first image and aplurality of second subsections in the second image; comparing each ofthe plurality of first subsections to a corresponding one of theplurality of second subsections; and automatically adjusting theexposure setting between the first image and the second image based on aresult of the comparison.
 36. The computer readable storage device ofclaim 35, wherein the processing device is configured to performoperations further comprising: minimizing a difference in brightnessbetween each of the plurality of first subsections and the correspondingone of the second subsections.
 37. The computer readable storage deviceof claim 31, wherein the processing device is configured to performoperations further comprising: generating a composite image byassembling the first image and the second image in response to thealigning; and displaying the composite image on the display device afterstoring the first image and the second image in the memory device. 38.The computer readable storage device of claim 37, wherein the processingdevice is configured to perform operations further comprising: enablingadjustment of at least one of luminance, chrominance, or hue associatedwith the first image or the second image in response to displaying thecomposite image on the display device.
 39. The computer readable storagedevice of claim 37, wherein the processing device is configured toperform operations further comprising: enabling replacement the firstimage or the second image with a third image in response to displayingthe composite image on the display device.
 40. The computer readablestorage device of claim 37, wherein the processing device is configuredto perform operations further comprising: displaying the composite imagein response to the second shutter release associated with the storing ofthe second image.